Apply Phi model with HuggingFace Causal ML

HuggingFace is a popular open-source platform that develops computation tools for building application using machine learning. It is widely known for its Transformers library which contains open-source implementation of transformer models for text, image, and audio task.

Phi-4-mini-instruct is a compact Microsoft language model optimized for instruction following and efficient deployment.

To make it easier to scale up causal language model prediction on a large dataset, we have integrated HuggingFace Causal LM with SynapseML. This integration makes it easy to use the Apache Spark distributed computing framework to process large data on text generation tasks.

This tutorial shows how to apply Phi-4 mini instruct at scale across standard prompts, chat templates, and GPU execution.

# %pip install --upgrade transformers==4.49.0 -q

model_name = "microsoft/Phi-4-mini-instruct"
ci_smoke = False
if "dbutils" in globals():
    dbutils.widgets.text("synapseml_ci_smoke", "false")
    ci_smoke = dbutils.widgets.get("synapseml_ci_smoke").lower() == "true"
generation_tokens = 10 if ci_smoke else 100

chats = [
    (1, "fix grammar: helol mi friend"),
    (2, "What is HuggingFace"),
    (3, "translate to Spanish: hello"),
]

chat_df = spark.createDataFrame(chats, ["row_index", "content"])
chat_df.show()

Define and Apply Phi-4 model

The following example demonstrates how to load the remote Phi-4 model from HuggingFace and apply it to chats.

from synapse.ml.hf import HuggingFaceCausalLM

phi4_transformer = (
    HuggingFaceCausalLM()
    .setModelName(model_name)
    .setInputCol("content")
    .setOutputCol("result")
    .setModelParam(max_new_tokens=generation_tokens)
    .setModelConfig(local_files_only=False, trust_remote_code=True)
)
result_df = phi4_transformer.transform(chat_df).collect()
display(result_df)

Apply Chat Template

from pyspark.sql.functions import udf
from pyspark.sql.types import ArrayType, MapType, StringType

reviews = [
    (1, "I like SynapseML"),
    (2, "Contoso is awful"),
]
reviews_df = spark.createDataFrame(reviews, ["row_index", "content"])

PROMPT_1 = f"""You are an AI assistant that identifies the sentiment of a given text. Respond with only the single word “positive” or “negative.”
        """


@udf
def make_template(s: str):
    return [{"role": "system", "content": PROMPT_1}, {"role": "user", "content": s}]


reviews_df = reviews_df.withColumn("messages", make_template("content"))

phi4_transformer = (
    HuggingFaceCausalLM()
    .setModelName(model_name)
    .setInputCol("messages")
    .setOutputCol("result")
    .setModelParam(max_new_tokens=10)
    .setModelConfig(local_files_only=False, trust_remote_code=True)
)
result_df = phi4_transformer.transform(reviews_df).collect()
display(result_df)

Use local cache

By caching the model, you can reduce initialization time. On Fabric, store the model in a Lakehouse and use setCachePath to load it.

# %%sh
# azcopy copy "https://mmlspark.blob.core.windows.net/huggingface/microsoft/Phi-4-mini-instruct" "/lakehouse/default/Files/microsoft/" --recursive=true

# phi4_transformer = (
#     HuggingFaceCausalLM()
#     .setCachePath("/lakehouse/default/Files/microsoft/Phi-4-mini-instruct")
#     .setInputCol("content")
#     .setOutputCol("result")
#     .setModelParam(max_new_tokens=1000)
# )
# result_df = phi4_transformer.transform(chat_df).collect()
# display(result_df)

Utilize GPU

To utilize GPU, passing device_map="cuda", torch_dtype="auto" to modelConfig.

phi4_transformer = (
    HuggingFaceCausalLM()
    .setModelName(model_name)
    .setInputCol("content")
    .setOutputCol("result")
    .setModelParam(max_new_tokens=generation_tokens)
    .setModelConfig(
        device_map="cuda",
        torch_dtype="auto",
        local_files_only=False,
        trust_remote_code=True,
    )
)
result_df = phi4_transformer.transform(chat_df).collect()
display(result_df)